1. Technical Field
The field of the invention is compression of a color image signal --such as that generated by a single sensor 3-color filter array-- whose chrominance-related (e.g., blue and red) components are sub-sampled with respect to its luminance-related (e.g., green) component.
2. Background Art
Electronic color cameras which employ only a single sensor are highly economical compared with electronic color cameras employing three separate sensors (e.g., red, green and blue sensors). A single image sensor (such as a CCD imager integrated circuit) can be made to produce a color image signal simply by imposing a color filter array over the sensor. The color filter array permits light of different colors to impinge on different picture elements (pixels) of the sensor in a fixed predetermined pattern. Typically one of the three colors (usually green) is most closely related to the luminance component of the image while the remaining two colors (usually red and blue) are most closely related to the chrominance components of the image. Using a single sensor necessarily reduces the resolution of each color component of the image, because sensor pixels dedicated to one color are in effect "missing" with respect to the other colors. Such "missing" pixels for a given color must therefore be inferred by interpolation in the reproduced image. Such interpolation can introduce into the reproduced image distortion objectionably visible to the human eye. The human eye is most sensitive to the luminance component of such distortion.
In order to minimize the perception of such distortion by the human eye and provide the most pleasing reproduced image, the resolution (or pixel density) of the luminance-related (e.g., green) pixels is increased at the expense of the chrominance-related (e.g., red and blue) pixels. Specifically, the color pattern of the color filter array is such that a majority of the sensor pixels receive the luminance-related (green) component of the light, while the remaining pixels receive the chrominance-related colors (e.g., red and blue). For example, a well-known color filter array (referred to as a 3G color filter array) consists of three rows of green pixels followed by a row of alternating red and blue pixels, so that 3/4 of the pixels are green and the blue and red pixels each comprise 1/8 of the pixels.
In reproducing a color image from the signal generated by the image sensor and color filter array combination and computing the missing chrominance-related pixels, the distortion perceived by the human eye is further reduced by interpolating between the ratios of the chrominance-related (red and blue) pixels to the co-located green pixels. This approach succeeds in reducing the distortion detected by the human eye because it can be shown that it reduces the luminance component of the distortion (without necessarily reducing the overall distortion).
A further improvement is achieved by performing such color interpolation of intensity values using the logarithm of each pixel intensity rather than the pixel intensity itself. This feature improves the color fidelity in the reproduced image because of the non-linear relationships involved in combining color signals, which are well-known and need not be described herein. One advantage is that this feature permits interpolation of color difference signals rather than ratios, since, for example, EQU log R/G=log R-log G EQU and EQU log B/G=log B-log G.
All of the foregoing color image signal processing methods are described in U.S. patent application Ser. No. 384,353 filed 24 Jul. 1989 by Yusheng T. Tsai, Kenneth A. Parulski and Majid Rabbani entitled "A COMPRESSION METHOD AND APPARATUS FOR SINGLE SENSOR COLOR IMAGING SYSTEMS" and assigned to the assignee of the present invention. The referenced application describes how to employ such methods in an image compression system. In the referenced patent application, compression prior to interpolation of the missing pixels is disclosed. In the image compression system, the amount by which digital data representing each spatial frequency component of the image signal is compressed is varied so as to compensate for the contrast sensitivity function of the human eye, as described in U.S. Pat. No. 4,780,761 to Scott J. Daly et al. and assigned to the assignee of the present invention.
The green signal is preferably interpolated linearly because the green signal is the one most closely related to the luminance component. One problem with the foregoing techniques is that the red, green and blue signals must be transformed to logarithms in order to best interpolate the color difference signals. This ultimately requires a multiplicity of such transformations, representing a significant processing burden.
Another problem is that image processing, such as sharpening or edge enhancement, can increase the visibility of distortions introduced into the reproduced image by the compression-decompression process. A related problem is that the modulation transfer function introduced by the image display (such as a color video monitor or color paper printing) can affect the visibility of distortions introduced into the image by the compression-decompression process.
Still another problem is that the human eye contrast sensitivity function has a lower frequency response to the color difference signals than to the luminance-related green signal. While the prior art teaches sub-sampling color pixels with respect to luminance pixels to accommodate for this aspect of the human visual system, a better method is needed. This is particularly true in the case of the 3G color filter array, in which sub-sampling of the red and blue pixels is non-isotropic, being two times greater along the columns of the array than along the rows of the array.
Yet another problem is that such edge enhancement processes tend to objectionably over-emphasize image features which are already sufficiently sharp. It has seemed that this is an unavoidable penalty which accompanies edge enhancement of the image. Therefore, there is a need for an edge enhancement process which does not over-emphasize sufficiently sharp image features. A related problem is that a CCD image sensor introduces CCD charge transfer noise into its output signal in a non-isotropic manner which affects high spatial frequency texture or edge features lying along one axis of the image more than those lying along the other axis of the image. Edge enhancement tends to emphasize such noise.
Accordingly, it is an object of the invention to reproduce an image from a single sensor/color filter array combination without requiring any logarithmic transformations whatsoever.
It is a further object of the invention to perform all processing and compression of all components of the image signal in the same color space, thereby minimizing the number of required transformations.
It is another object of the invention to automatically compensate for the effects of the edge enhancement process and of the image display on the visibility of errors introduced by the compression/decompression image process.
It is still another object of the invention to provide different adaptive corrections in the compression-decompression processes for different (luminance-related and chrominance-related) components of the image best suited for the respective components.
It is a still further object of the invention to provide different adaptive corrections in the compression-decompression processes for different components of the image in accordance with the lower frequency response of the human visual system to the color difference signals and in accordance with the non-isotropic pattern of the green, red and blue pixels of the color filter array.
It is yet another object of the invention to provide an edge enhancement process which does not over-emphasize image features which are already sufficiently sharp and which does not require any transformation of the image data to another color space (e.g., a log or anti-log transformation). It is a related object of the invention to provide such an edge enhancement process which refrains from emphasizing CCD transfer noise affecting high spatial frequency image features along one axis without detracting from the edge enhancement of features lying along the other axis of the image.